Skip to main content

How to Backfill Azure FOCUS Export Data

Overview​

Azure FOCUS exports typically generate data going forward from the date of creation. However, you may need to retrieve historical cost data for analysis or reporting purposes. This guide explains how to backfill your Azure FOCUS export data by manually triggering exports for specific date ranges.

Why Backfill FOCUS Export Data?​

Backfilling is useful when:

  • You've recently set up FOCUS exports and need historical data
  • There are gaps in your export data due to configuration issues
  • You need specific historical periods for cost analysis
  • You're migrating from another cost management solution

Prerequisites​

  • An existing Azure FOCUS export configured in your Azure portal
  • Appropriate permissions to manage exports in Azure Cost Management + Billing
  • Access to the storage account where exports are stored

Steps to Backfill FOCUS Export Data​

1. Navigate to Your Existing Export​

  1. Log in to Azure Portal: Azure Portal
  2. Go to Cost Management + Billing
  3. Select the appropriate billing scope
  4. Navigate to Exports in the left sidebar
  5. Click on your existing FOCUS export

2. Trigger Manual Export for Specific Dates​

  1. Select "Export selected dates":

    • In the export details page, look for the "Export selected dates" option
    • This allows you to manually trigger exports for specific time periods

  2. Configure Date Range:

    • Start Date: Select the beginning of the period you want to export
    • End Date: Select the end of the period
important

Note that Azure only allows running 1 export at a time, month by month

  1. Trigger the Export:
    • Click "Execute" to start the export process
    • The export will be queued and processed by Azure

3. Monitor Export Progress​

  1. Check Export Status:

    • Return to the Exports list to monitor the status
    • Exports typically show as "In progress" or "Succeeded"

  2. Verify Data in Storage Account:

    • Once completed, navigate to your storage account
    • Check the nops/billing directory for the exported files
    • Files will be organized by date in the Parquet format, under the export name and following the format 20240501-20240531 for example.

Important Considerations​

Processing Time​

  • Backfilling can take several hours depending on:
    • The amount of data in the selected period
    • The number of resources and subscriptions
    • Current Azure service load

Data Availability​

  • Azure typically retains detailed billing data for the past 13 months
  • Data older than 13 months may not be available for export
  • Some detailed metrics may only be available for more recent periods

Storage Costs​

  • Backfilling will increase storage usage in your Azure storage account
  • Consider the additional storage costs when backfilling large date ranges

nOps Synchronization​

  • nOps automatically checks for new export files every 24 hours
  • Backfilled data will be picked up during the next synchronization cycle
  • Large amounts of historical data may take additional time to process in nOps

Best Practices for Backfilling​

  1. Start with Recent Data:

    • Begin backfilling with the most recent months
    • Work backwards to ensure critical recent data is available first

  2. Process Month by Month:

    • Export one month at a time rather than large date ranges
    • This makes it easier to identify and resolve any issues

  3. Monitor Storage Account:

    • Regularly check that exports are completing successfully
    • Verify file sizes are reasonable for the period exported

  4. Schedule During Off-Peak Hours:

    • If possible, trigger backfill exports during off-peak hours
    • This can result in faster processing times

  5. Document Your Progress:

    • Keep track of which periods have been successfully backfilled
    • Note any periods that failed or need to be re-exported

Next Steps​

After successfully backfilling your Azure FOCUS export data:

  1. Verify in nOps: Check that the historical data appears in your nOps reports after the next synchronization
  2. Set Up Monitoring: Configure alerts for export failures to prevent future data gaps
  3. Regular Validation: Periodically verify that ongoing exports are completing successfully

For additional assistance with Azure FOCUS exports or nOps integration, refer to the Azure FOCUS Export Configuration guide or contact nOps support.